Skip to content

test: property-check plugin health classification and report shape#596

Open
ndycode wants to merge 2 commits into
mainfrom
claude/audit-77-health-property
Open

test: property-check plugin health classification and report shape#596
ndycode wants to merge 2 commits into
mainfrom
claude/audit-77-health-property

Conversation

@ndycode

@ndycode ndycode commented Jun 11, 2026

Copy link
Copy Markdown
Owner

Summary

  • Adds a fast-check property suite for lib/health.ts — the pure shape-transformation behind codex-multi-auth health reporting (AUDIT-M08/D-04 documents it as deliberately decoupled from the manager, which makes it an exact-oracle target).

What Changed

New test/property/health.property.test.ts (3 properties over getAccountHealth/formatHealthReport, circuit registry cleared per iteration since getAccountHealth reads the module-level breakers):

  1. Classification oracle — for any pool (0–8 accounts; timestamps straddling now so both sides of the rate-limit/cooldown comparisons generate; undefined exercising the ?? 0 fallbacks; health 0–100), every per-account flag, all three counts, and the status partition derive exactly from the documented rule: healthy ⇔ not rate-limited ∧ not cooling ∧ health >= 50 ∧ circuit closed. The status edges are pinned too — an empty pool reads healthy, zero healthy of some reads unhealthy, anything between reads degraded.
  2. Circuit integration — tripping the documented account:<index> fallback breaker (3 failures) for any subset of identity-less accounts shows up in circuitState, disqualifies otherwise-perfect accounts from healthyAccountCount, and drives the status partition.
  3. Report shapeformatHealthReport names every account (email or the Account N fallback) with its health percentage and exactly the flags its classification implies, and the Rate Limited:/Cooling Down: summary lines appear iff their counts are nonzero.

No SUT bugs found.

Validation

  • npm test -- test/property/health.property.test.ts test/health.test.ts — 17/17 (new 3 + existing 14 untouched)
  • npm run typecheck (also via pre-commit hook)
  • npx eslint test/property/health.property.test.ts --max-warnings=0
  • Module-state hygiene: clearCircuitBreakers() in before/afterEach and at each property iteration start

Docs and Governance Checklist

  • Test-only; no behavior or docs surface changed

Risk and Rollback

  • Risk level: minimal — additive test file; conventions match the existing test/property/ suites.
  • Rollback plan: revert the single commit.

https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB


Generated by Claude Code

note: greptile review for oc-chatgpt-multi-auth. cite files like lib/foo.ts:123. confirm regression tests + windows concurrency/token redaction coverage.

Greptile Summary

additive test-only pr adding test/property/health.property.test.ts, a 4-property fast-check suite for lib/health.ts's getAccountHealth and formatHealthReport. no sut code is touched.

  • classification + counts oracle — property 1 derives every per-account flag, the three aggregate counts, and the status partition from first principles against the documented rule (healthy ⇔ not rate-limited ∧ not cooling ∧ health ≥ 50 ∧ circuit closed).
  • circuit integration — property 2 now covers both open and half-open disqualification via vi.useFakeTimers() + canExecute() forcing the transition, closing the gap flagged in the prior review round.
  • email aliasing + report shape — property 3 explicitly asserts shared-key propagation; property 4 checks the full formatted report including Rate Limited: / Cooling Down: summary lines. per-account line format for circuit-flagged accounts is not exercised in property 4 (see inline comment).

Confidence Score: 5/5

safe to merge — additive test file only, no sut or production code changes

the change is a single new test file with no modifications to lib/ or any other production code. module-state hygiene is handled correctly via clearCircuitBreakers() in before/afterEach and at each property iteration start. the half-open path, previously unexercised, is now covered with proper fake-timer discipline and a finally-block cleanup. the only gap is a minor coverage omission in property 4 (circuit flag per-account line format), which does not affect production correctness.

test/property/health.property.test.ts — the report-shape property could be strengthened to cover per-account lines that include circuit flags

Important Files Changed

Filename Overview
test/property/health.property.test.ts adds 4 fast-check properties covering health classification, open/half-open circuit disqualification, email-key aliasing, and report formatting; per-account circuit flag formatting is not validated in the report-shape property

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[arbAccounts generator] --> P1[property 1: classification oracle]
    A --> P4[property 4: report shape]

    P2_arb[fc.integer count\nfc.uniqueArray trippedRaw\nfc.boolean probeHalfOpen] --> P2[property 2: circuit integration]

    P2 -->|probeHalfOpen=false| OPEN[circuits stay open\nexpectedState = 'open']
    P2 -->|probeHalfOpen=true & tripped>0| HALF[vi.setSystemTime + canExecute\nexpectedState = 'half-open']

    FC_BOOL[fc.boolean shareEmail] --> P3[property 3: email aliasing]

    P1 --> SUT1[getAccountHealth accounts NOW]
    P4 --> SUT1
    P2 --> SUT1
    P3 --> SUT1
    SUT1 --> SUT2[formatHealthReport]

    SUT1 --> ASSERT_COUNTS[assert counts + status partition]
    SUT2 --> ASSERT_REPORT[assert per-account lines + summary flags]
    P2 --> ASSERT_CIRCUIT[assert circuitState per account\nassert healthyAccountCount\nassert report.includes circuit-state]
Loading

Fix All in Codex

Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
test/property/health.property.test.ts:197-233
**circuit flag line format never validated under property 4**

property 4 always calls `clearCircuitBreakers()` and uses a fresh registry, so `account.circuitState` is always `"closed"` for every generated account. the assertion `expect(line?.includes("circuit-")).toBe(false)` only ever exercises the false branch; the format of per-account lines when a circuit flag is present (e.g. `circuit-open` or `circuit-half-open`) is never checked here. property 2 validates that the string `circuit-${expectedTrippedState}` appears somewhere in the full report, but not that it is correctly embedded in the right account's detail line. a mutation that emits `circuit_${state}` or that attaches the flag to the wrong account row would escape property 4 entirely, and property 2's `report.includes(...)` check would only catch the prefix change. extending property 4 to occasionally trip a circuit for one account — or splitting the "with-circuit" case into a small companion property — would close the gap.

Reviews (2): Last reviewed commit: "test: exercise half-open disqualificatio..." | Re-trigger Greptile

Three fast-check properties over getAccountHealth/formatHealthReport:

- counts and status derive exactly from the per-account classification
  for any pool (timestamps straddling now, undefined fallbacks, health
  0..100): healthy iff not rate-limited, not cooling, health >= 50,
  circuit closed; status partitions empty->healthy, none->unhealthy,
  some->degraded, all->healthy
- an open circuit disqualifies an otherwise perfect account: tripping
  the account:<index> fallback breaker for any subset is reflected in
  circuitState, subtracted from healthyAccountCount, and drives the
  status partition
- the formatted report names every account with its health percentage
  and exactly the flags its classification implies, and the summary
  lines appear iff their counts are nonzero

Companion to the property suites in #574/#575/#592-#595.

https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@ndycode, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 20 minutes. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more credits in the billing tab to continue.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 128db75e-16a9-4839-9fe1-de79ad26bb56

📥 Commits

Reviewing files that changed from the base of the PR and between c255e14 and aa26f68.

📒 Files selected for processing (1)
  • test/property/health.property.test.ts
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/audit-77-health-property
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch claude/audit-77-health-property

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Comment thread test/property/health.property.test.ts Outdated
Comment thread test/property/health.property.test.ts
Greptile flagged two coverage gaps: the half-open state was unreachable
without advancing the clock, and accounts sharing an email silently
shared a breaker without an explicit contract. Property 2 now probes
open -> half-open under fake timers (both states disqualify, and the
report renders the precise circuit flag), pool emails are unique by
construction, and a dedicated property pins shared-email accounts to
one breaker while distinct emails stay isolated - derived through the
same getAccountIdentityKey the SUT uses.

https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants